Deep neural networks are an integral part of machine learning and data science toolset for practical data-driven problem solving. With continual miniaturization ever more applications can be found in embedded systems. Common embedded applications include on-device image recognition and signal processing. Despite recent advances in generalization and optimization theory specific to deep networks, deploying in actual embedded hardware remains a challenge due to storage, real-time throughput, and arithmetic complexity restrictions [He et al., 2018]. Therefore, compression methods for achieving high model sparsity and numerical efficiency without losing much in performance are especially relevant.